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PROBLEM TO BE SOLVED: To provide a suitable video 
retrieval device by using it for speedily detecting a video 
which a viewer desires from a large quantity of spread 

videos. 

SOLUTION: A change position detection part 13 detects 
the switch of a scene. A representative picture extraction 
part 14 extracts one to several pieces of representative 
picture VS(video software) from the respective scenes. A 
character data extraction part 23 extracts character data 
from them. Character data is accumulated in a character 
data accumulation part 23 with information on the 
extracted position. When a user desires to check 
something in business or personally, accumulated 
character data is retrieved with a keyword showing it. 
The representative picture VD which is hit and original video software VS are displayed on a 
display 26. Data can be selected from the display of a table. 




LEGAL STATUS 

[Date of request for examination] 1 5.07.2004 

[Date of sending the examiner's decision of 13.02.2007 
rejection] 

[Kind of final disposal of application other than 



http://wwwl9.ipdl.inpit.go.jp/PAl/result/detail/main/wAAAi8aGNkDA41 1039343P1 .htm 1 1/26/2007 



Searching PAJ 



Page 2 of 2 



the examiner's decision of rejection or 
application converted registration] 

[Date of final disposal for application] 

[Patent number] 

[Date of registration] 

[Number of appeal against examiner's 

decision of rejection] 

[Date of requesting appeal against examiner's 
decision of rejection] 

[Date of extinction of right] 



http://wvml9jpdlJnpit.go jp/PAl/result/detaiVmain/wAAAi8aGh^ 1 1/26/2007 



JP,n-039343,A [CLAIMS] 



Page 1 of I 



♦ NOTICES • 

JPO and INPXT axe not responsible for any 
damages caused by the use of this translation. 

1. This document has been translated by computer. So the translation may not reflect the original 
precisely. 

2. **** shows the word which can not be translated. 
3.1n the drawings, any words are not translated. 



CLAIMS 



[Claim(s)] 

[Claim 1] Image retrieval equipment characterized by having an alphabetic character data extraction 
means to extract alphabetic data from image software, an alphabetic character data accumulation means 
to accumulate said alphabetic data with the extract positional information, a selection means to choose a 
desired thing out of said accumulated alphabetic data, and the cash-drawer means that pulls out the 
image conceming the this chosen alphabetic data out of said image software. 
[Claim 2] Image retrieval equipment according to claim 1 with which said alphabetic character data 
extraction means is characterized by extracting said alphabetic data from the representation image 
extracted from said image software. 

[Claim 3] Image retrieval equipment according to claim 1 or 2 characterized by said cash-drawer means 
pulling out the representation image conceming said selected alphabetic data. 



[Translation done.] 
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* NOTICES * 

JPO and XNPIT are not responsible for any 
damages caused by the use of this translation. 

1 .This document has been translated by computer. So the translation may not reflect the original 
precisely. 

2,**** shows the word which can not be translated. 
3. In the drawings, any words are not translated. 



DETAILED DESCRIPTION 



[Detailed Description of the Invention] 
[0001] 

[Field of the Invention] This invention is used for finding quickly out that which the viewer wants out of 
the image circulated in large quantities in detail about image retrieval equipment, and relates to suitable 
image retrieval equipment. In addition, on these specifications, the thing equivalent to one television 
broadcasting is called a "image" or "screen", and what put two or more [ of these ] in a row is called 
"image." What added the sound signal, the caption code (title sign), etc. to this is called "image software 
( drawing 3 VS (this sign VS omits hereafter))." These are supplied with a motion-picture film, a video 
tape, television broadcasting, DVD (a digital videodisc or digital bar SATAIZU disk), the Internet, a 
computer storage, etc. 
[0002] 

[Description of the Prior Art] A movie, a sport relay broadcast, etc. are the time when image software is 
supplied in large quantities. Otherwise, the image software of very various fields, such as a drama, 
music, culture, and a report, exists, and it is expected that the amount of supply will increase fiirther 
fi-om now on. The utility value of fliese images software is very high as it is also in the proverb of seeing 
is believing, make it the thing of work — make it individual hobby level — the value will be very high, if 
these are grasped exactly and it can use as a library as the occasional decision ingredient. 
[0003] Image software meets the passage of time amount. People need time amount, in order to see this. 
When the amoimt of supply becomes huge in this way, the image transcription etc. is carried out to the 
video tape like before, and it becomes impossible so, to say that it sees slowly later. Then, the keyword, 
the index for retrieval, etc. are attached to a required image (fimne) and a required image, for example, 
and how to search later using this can be considered. 
[0004] 

[Problem(s) to be Solved by the Invention] However, this approach requires time amoimt and time and 
effort for the input of a keyword, an index, etc. Although the contents are checked about each image 
software and a keyword, an index, etc. are specifically attached for these whole or each image of every, 
aside fi-om the ancient times which had little amoimt of supply, such an art is used in the fixture which is 
called multimedia age and whose amount increases today or fiirther, and does not have ♦*♦***, either. It 
always becomes a technical problem an index [ what kind of keyword and ] are attached. 
[0005] Along with improvement in the calculation speed of a computer, it came to pull out what "a fiill- 
text search (fiiU-text search) fiiU-text search" Carries out the alphabetic data text itself, and corresponds 
by the information retrieval centering on an alphabetic character, without carrying out registration of 
indexation or a keyword beforehand to alphabetic data. In this case, if the object of the noun in the text 
is carried out, it will be known that quite exact retrieval can be performed, and if a user inputs the 
language currently used every day, the text containing that language will be searched at high speed. 
[0006] It is possible like this that image retrieval also carries out "all image retrieval (ftiU image 
search)." However, it is the processing that whose a match is looked for about an image itself time 
amount requires considerably, and it is common for it to be meaningless. For this reason, a certain 
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option is required. 

[0007] The purpose of this invention is to use it for finding out quickly that which the viewer wants, and 

offer suitable image retrieval equipment out of such an image circulated in large quantities. 

[0008] 

[Means for Solving the Problem] By invention of claim 1, an alphabetic character data extraction means 
to extract alphabetic data from image software, an alphabetic character data accumulation means to 
accumulate said alphabetic data with the information on the extract location, a selection means to choose 
a desired thing out of said accumulated alphabetic data, and the cash-drawer means that pulls out the 
image concerning the this chosen alphabetic data out of said image software are used for the above- 
mentioned purpose achievement. 

[0009] Moreover, in invention of claim 2, said alphabetic character data extraction means extracts said 
alphabetic data from the representation image extracted from said image software. Moreover, in 
invention of claim 3, the cash-drawer means said to claim 1 or claim 2 pulls out the representation image 
concerning said selected alphabetic data. 

[0010] This invention is observing the alphabetic character contained in image software. Here, the 
alphabetic character in image software has the following descriptions. 

(1) It is used for an alphabetic character telling important information, such as a title and a performer, on 
the movie. 

(2) The title of a foreign movie is made so that conversation and a story may be told in the minimum 
alphabetic character. 

(3) The alphabetic character is used for arranging the outline of an incident by the news of TV 
(television). The alphabetic character is used here in order to adjust information in a compact. 

(4) As for the result of a horse race etc., numerical information is displayed on a screen in many cases as 
a result of a sport. 

[001 1] Thus, the alphabetic character in it is used for transfer of information with explicit semantics to 
the image software itself having told delicate feeling and sensuous things. By this invention, paying 
attention to this alphabetic data, this alphabetic data is extracted and accmnulated by the configuration 
given in a claim, and facilities are given to a viewer by so to speak considering as the database of an 
image and an image. 
[0012] 

[Embodiment of the Invention] Hereafter, it explains based on the example of a gestalt of illustration 
implementation of the detail of this invention. The outline of introduction and the example of a gestalt of 
operation ( drawing 1 image retrieval equipment 100) is explained. 

(1) Original image software, such as received TV broadcast (television broadcasting), is supplied and 
accumulated in the original image are recording section 12 through the input interface 1 1 (although the 
original image are recording section 12 should consider as the "Hara image software are recording 
section*' correctly, since it was redimdancy, it omitted "software"), moreover, the change location 
detecting element 13 is also supplied — having — a scene (a scene, cut) — changing — etc. — the location 
where the condition of image software changes is detected. Similarly the image software supplied by 
another gestalt, for example, a video tape, DVD, the Intemet, CD-ROM, etc. is incorporated from the 
input interface 11. 

[0013] (2) If it is detected that the change of state, for example, a scene, changed by the change location 
detecting element 13, the representation image extract section 14 will extract the representation image 
VD ( drawing 2 (following and sign VD abbreviation)) following this. A representation image is the 
static image of one sheet or the short dynamic image for about 2 - S seconds taken out from near [ the ] a 
change-of-state location, and the set is accumulated here as an epitome VE ( drawing 2 (following sign 
VE abbreviation)). An epitome becomes the about 1/ten to 1/100 amount of data compared with original 
image software. Therefore, next processing becomes simple. It is used also as a decision ingredient of 
whether to see original image software. 

[0014] - A representation image may be extracted from original image software a fixed period, the 
alphabetic character in image software is comparatively long — it is indicated by time amoimt. This is 
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because time amount until a viewer finishes reading the displayed text is taken into consideration, and if 
it says conversely, the same alphabetic character is displayed in the meantime, therefore, if it is alike and 
a still picture or an animation is extracted at the rate of one sheet, this time interval, for example, 5 
seconds, 10 seconds, and the alphabetic character contained in that image software can carry out a 
***4i*4i*4t4i4t«*4i* extract. When based on this technique, it is not necessary to extract a representation 
image, and an image is extracted with a fixed lime interval fi-om a original image, and the alphabetic 
character contained in it is extracted. In addition, it may be accumulated and the image extracted here 
may also be used as an epitome. 

[0015] (3) Extract an alphabetic character from a representation image. Character recognition software is 
used and alphabetic data is picked out from the extracted representation image. The taken-out alphabetic 
data is accumulated as a keyword. It is also good to limit this keyword only to a noun depending on the 
case. In case a keyword is begim and said epitome and original image software are accumulated, the 
channel number, the broadcast date, the time second, the fimie number, etc. attach the header 
information which can maintain mutual link relation. 

[0016] The accumulated keyword, an epitome, and original image software are used as the following. 

(1) Drive in as a keyword the language which expresses the event directly, for example, the language 
like the free keyword of PATOLIS, at every need about the theme against which the viewer bears an 
interest by the individual thing, concerning work to know whether related image software is 
accumulated. 

(2) It collates with the alphabetic data with which this was accimiulated. A viewer sets up the Ruhr of 
retrieval of ftiU coincidence and partial coincidence and others. If there are some corresponding, next the 
epitome corresponding to it will be taken out and it will check the contents of the image. 

(3) An epitome is seen, if it is thought that it is a thing about the theme for which it asks, the original 
image software will be taken out and the contents will be checked. 

[0017] An epitome is the magnitude of 1/10 to 1/100 of a original image as above-mentioned, and 
seeing this does not take the time amount like ♦*. Therefore, there is little time and effort which requires 
for retrieval the direction to which made the two-step style in this way, and it was presupposed that it 
discems whether it is that for which it asks in an epitome first of all, and it ends. 

[0018] When there are some which register a keyword beforehand and correspond to this, it is also good 
to record independently the table-of-contents information about the image software etc. For example, an 
one day viewer checks an are recording condition for this. The appropriate thing is displayed on a screen 
by list, a viewer is made to choose and the epitome and original image software are displayed. A 
keyword is not inputted, but the alphabetic data acciunulated is displayed by the order of the Japanese 
syllabary, and a viewer chooses a desired thing out of it, and may be made to display the epitome and 
original image software. 

[0019] Hereafter, the detail of the image retrieval equipment 100 of the example of a gestalt of operation 
is quoted, and drawing is explained. That is, the image software supplied from TV broadcast receive 
section 16, the DVD player 17, the video tape deck 18, the Internet cormectivity section 19, and CD- 
ROM drive 20 and otiiers is incorporated by image retrieval equipment 100 through the input interface 
11 in drawing 1 . 

[0020] TV broadcast receive section 16 has many receiving channels. For example, terrestrial TV 
broadcast (VHF, UHF), satellite broadcasting service, a cable TV, etc. receive the analog and digital TV 
broadcast for which a viewer asks about two or more channels to two or more coincidence. From the 
Internet connectivity section 19, the image software on the Internet for which it asks is periodically 
incorporated following actuation of a viewer. 

[0021] The image software incorporated from these is stored in the original image are recording section 
12. Are recording is good to carry out by digitizing. The thing of an analog form is digitized with the 
input interface 11, and is accumulated in the original image are recording section 12. In order to stop the 
amount of data, it is good to carry out a data compression by the method of MUSE, MPEG, JPEG, and 
others. The image software on digital television broadcast and the Internet is good at the Image 
software is accumulated in the original image are recording section 12 as a break and every one file for 



http://www4.ipdl.inpit.go.jp/cgi-bin/tran__web_cgi_ejje 



11/26/2007 



JP,1 1-039343,A PETAILED DESCRIPTION] 



Page 4 of 10 



example, per 1 0 minutes. Each receiving date, the receiving time of the initial data of a file, a channel 
number, etc. record table-of-contents information (index) required for next read-out on a file allocation 
table (FAT). 

[0022] Storing in the original image are recording section 12 is continuously carried out to some extent 
like 72 hours about the specified channel. Speech information is also recorded. In addition, a viewer sets 
up the date which records, time of day, and the channel for record by the user control unit 22. There are 
some time zones which turn out that it is not necessary to record from the start, and there are also some 
programs in which a viewer does not get interested. Then, not only at time but at the time, if it is made 
even a part unit is fine and possible [ a time zone setup ] for it, the next contents check activity of record 
will become easy. A time zone configuration procedure is the same as the existing video tape recorder 
etc., and good. 

[0023] record (are recording) — being cyclic (cyclic) - it is good to carry out. For example, record of the 
memory capacity of the original image are recording section 12 will be enabled for five-channel and 
continuation three days. When there is time amount, a viewer checks the alphabetic data accumulated, 
an epitome, and original image software, and eliminates a unnecessary thing. The storage region of the 
original image are recording section 12 and the epitome are recording section 21 is vacant now. Storage 
capacity may be lost by a certain thing [ that carry out period continuation and the contents of broadcast 
are recorded ]. in this case, the inside of a recorded image — most — being old ~ although — a top — the 
following new video signal — overwrite — it carries out. 

[0024] What is necessary is for there to be no need of specifying the date time of day to record, and to 
specify only the channel (channel which wants to supervise) for which it asks by the user control unit 
22, if it carries out like this. But there is a time zone which does not need to be recorded from the start as 
mentioned above. Therefore, even when recording cyclically, there is little storage capacity which the 
direction whose setup of the time zone of record needlessness was enabled consumes, and it ends. 
[0025] Retum and the change location detecting element 13 will notify that to drawing 1 at the 
representation image extract section 14, if a change of state arises to the attention element with which it 
was specified in the television broadcasting then incorporated or the DVD receipt image software 
currently then reproduced. 

[0026] This invention extracts the alphabetic data contained in image software, accumulates this, and 
utilizes it as a key of image retrieval. Alphabetic data may be soon extracted from the original image 
software, as mentioned above, but ( drawing 1 broken line L) as mentioned above, in order to make the 
check of the contents of record easy, and in order to simplify processing, as shown in drawing 2 , 
extracts the representation image VD from original image software, and extracts alphabetic data out of 
this representation image VD here (the sign VD of a representation image etc. is omitted suitably 
below). 

[0027] Detection of a change-of-state location is for giving the motive of this representation image 
extract, and the representation image extract section 14 extracts a representation image following this. 
Moreover, not only the still picture of one sheet but the animation for about several seconds concerning 
the change location shall be contained in each representation image. 

[0028] Detectable various elements (description) are contained in image software (program). For 
example, the element in which the sound of people's voice, music, a cheer, and pulse nature and much 
other detection are possible exists in voice (sound). Moreover, some detectable elements ~ that the 
brightness of a screen changes suddenly and the color of the whole screen changes [ standing it still or ] 
suddenly ~ exist in an image. A representation image extract here is performed in order to take out the 
alphabetic data which exists in image software. Therefore, it is good to choose the element which the 
change produces in relation to the appearance of the new alphabetic character to a screen also about the 
element to which its attention is paid. 

[0029] For example, after in the case of a news program reading of the news manuscript by announcer 
and the axle-pin rake is completed and voice breaks off, a screen projects a new alphabetic character in 
the form which the following news manuscript read out and synchronized with initiation. Therefore, 
change of an element called intemiittence of this voice can also be used for the motive of a 
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representation image extract. The inclination for an educational program etc. to be the same is seen. 
[0030] Change of an element and the relevance of an alphabetic character appearance change according 
to the genre (field) of image software. So, if the property of the image software for are recording at that 
time is clear from a race card etc., in case said channel for record is set up, the element to which its 
attention is paid is made into which, or it carries out as [ set / a viewer ]. 

[0031] There is also an element independent of a property or a field. When saying that the object of 
record is not specified, i.e., it records continuously about some channels etc. and the alphabetic data in it 
is taken out as a search key, it is good to pay one's attention to such a universal element. The example is 
shown below. 

(1) Cut down simply the image at the time of logging Hara image software initiation of a head image as 
a title (representation image). When the broadcast start time of the program concerned can be grasped by 
a race card etc., the head image is extracted. For example, if it divides per 10 minutes and prolonged 
continuation record is carried out, the head image of each break will be extracted as a title. 
[0032] (2) Cut down an image periodically for every fixed time amount from periodic logging image 
software. If the epitome which consists of the representation image of M sheets is generated about the 
original image software which consists of images of N sheets, the image in every N/M sheets will be 
taken out from original image software as a representation image. This is the simplest approach. In 
addition, by the picture compression technique, such as MPEG, JPEG, and MUSE, a perfect image is 
transmitted to how many sheet cage, and difference with a front image is transmitted in the meantime. 
Therefore, about such image data, the perfect image sent to several of the sheets one sheet may be 
extracted as a representation image. 

[0033] When taking out alphabetic data from image software soon, the technique of this periodic 
logging is usefiil without generating an epitome. That is, although the alphabetic character in image 
software is displayed since it reads to a viewer and is surely given, it requires appropriate time amount 
for people reading a thing. For this reason, the alphabetic character of a screen is also displayed in 
consideration of it comparatively long time amount, for example, S seconds, 10 seconds, and often. 
Therefore, if in the case of television for S seconds then an image will be taken out and inspected at the 
rate of one sheet to 300 frames supposing one sheet is displayed for 10 seconds every 150 frames, 

of an alphabetic character which appears in the image software can be extracted. 
[0034] So, when based on this technique, an epitome is not generated but the direct alphabetic character 
extract from original image software is perfomied. But the epitome is also the decision ingredient of 
whether to see original image software as above-mentioned. Therefore, alphabetic data is considered as 
the direct extract from the original image said here, the element which will generate the epitome which, 
on the other hand, expresses the contents of the image software concerned exactly reasonable is chosen 
independently, the change of state is made into a motive, and you may make it generate the epitome of 
the image software. 

[0035] In this case, there is the following as an example of the element to which its attention is paid. 
Genre of an image Attention element news Cut drama with a pattem (flip) Cut with a title Cut 
documentary with voice Cut English conversation with voice Cut sport with a title Applause, the cut 
which a cheer arises, and its circumference (voice Climax) 

Animation Cut with a title Cut which is standing it still for a long time Cut TV shopping with voice Cut 

with a title (infomiation, such as a price, is in sight) 

Song program Cut from which music begins (from voice to distinction) 

Educational program Cut variety show with a pattem Cut orchestra which a cheer arises Place which 
music starts (periodic extract) 

Weather report Stationary cut (pattem (flip) = plate which drew an alphabetic character and drawing.) A 
speaker etc. uses it by TV. 

In addition, an attention element, a predetermined location, etc. change with properties of a program and 
a broadcast channel in this way. Then, it is good to perform epitome generation and a check several 
times and to decide a desirable attention element, a predetermined location, etc. in it. 
[0036] (3) When the logging scene for every scene changes, say in many cases that it projects an 
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alphabetic character new in an image. Therefore, it is also good to detect this, to take out the 
representation image of number of sheets suitably for every scene, and to investigate the existence of 
alphabetic data. Although it is arbitrary fixjm which location in a scene a representation image is taken, 
supposing a scene changes by CSl and CS2 of drawing 3 , for example, the center PC of a scene etc. 
will become one candidate with them. 

[0037] Distinction from which a scene changes is performed as the following. 

(1) The intensive processing image of a pixel has two-dimensional breadth needless to say. For example, 
NTSC system is the assembly of a pixel (a pixel, point) which is about called 250 dot x525 line, there 
are lightness, saturation, and a hue in each pixel, and 30 images which consist of a set which is such a 
pixel are transmitted in 1 second. 

[0038] Needless to say, there are many pixels. Then, concentration-ization of a pixel is performed here 
for improvement in the speed of processing, and simplification. For example, if image data will become 
1/16 of amoimts and same processing will be carried out about the pixel of 8x8 supposing it calculates 
an average value in the sum of the pixel of every direction 4x4, image data will become 1/64. 
[0039] This concentration-ization abstracts a loose change of an image and has the operation which 
makes easy distinction from which a scene changes. That is, a scene changes, distinction measures 
fundamentally the pixel and pixel of two images which get mixed up, and it discerns whether a big 
difference is there. If the data of both pixel are compared in some fields of an image and many are 
specifically common, I will think that it is an image only in one scene which the person moved, without 
a background changing there. 

[0040] however, for example, ** zoom-in / down, and ** — when there is modification of the sense of a 
camera made slowly and the pixel comparison of the image of order is carried out without carrying out 
intensive processing here, even when appearance is slight — pixel level — the part of ****** — data — 
being different — actual — a scene — on tfie way — it comes out, and although it is, a scene will change 
and these many will be judged accidentally [ be / it / a location ]. 

[0041] If the same thing is looked at on the concentration-ization-processed coarse pixel level, since the 
pixel is coarse, the image to a certain number of sheets has the same data, and is changeless. So, these 
are correctly judged to be an image belonging to the same scene. Then, they are the value of these coarse 
pixels, and a (tx, y) as pretreatment first. 

It takes out here. It is trtime of day here, x y: Coordinate of the image after intensive processing a: Value 
of the color of the point (x y), a — R, G, and B — the — or it puts in ** picking — CI, R+C2, G+C3, andB 
— then, good (as for R, G, and B, the value of three-primary-colors information, and CI, C2 and C3 are 
weighting factors). 

[0042] (2) Ask for the difference between each of the pixel and each pixel of the image before that for 
every image about the image data which performed concentration-ized processing of the direction of 
time amount, next the above-mentioned concentration-ized processing. And it judges whether it is 
magnitude to the extent that you may judge that a scene changes and the difference is a location. If you 
want to simplify processing, the threshold fixed about the size of the difference at this time is defined. 
The change rate of a scene judges with the image having changed there about a certain number in the 
coarse pixel which constitutes an image, when the difference was over the threshold. 
[0043] But even if it belongs to the same scene, the magnitude of the difference between pixels changes 
with the contents. If 1 coma 1 coma is the scene which changes early, the difference of the image of 
order is large, and difference is small if late. Therefore, although processing simple like the above is also 
law on the other hand, here adds the following processing fiulher in order to extract a representation 
image here more exactly. 

the time difference of "image - comparative processing" - data of each point which constitutes the 
image of one sheet It asks for time difference about a (tx, y). Namely, d(tx, y) =a(tx, y)-a (t- ** tx, y) 
********. However, **t: It is proper time amount width of face. This shows the difference between 
pixels of the image of two sheets with which only **t adjoins each other in time (variation). 
[0044] It asks for this difference d (tx, y) about a predetermined period, for example, each image for 1 
minute. It will be [Equation 1] if expressed with a formula. 
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D a(t)= Ed(t .x.y) 
It becomes. 

[0045] Drawings is the example of the difference Da about the concentration-ized image for 1 minute 
(t) for which carried out in this way and it asked. The location CSl and CS2 where a value is large, i.e., 
the locations where the difference of an image is large, means that an image has a certain big change 
there, and possibility that the scene has changed is high there. Therefore, between these CSl and CS2 is 
presumed to be one scene, and the representation image of number of sheets is suitably extracted out of 
this, the time difference of "image which these say to the above-mentioned header - it is comparison 
processing." 

[0046] In addition, it asks for the difference accompanying a change of a screen from this, and you may 
make it detect the change rate of a scene about each frame which constitutes image software paying 
attention to the video signal of horizontal scanning line 1 duty for example, near an image center. The 
video signal of this horizontal scanning line 1 duty is divided into the section of N individual, and, 
specifically, the sum or the average is calculated about each section. And it asks for the difference to the 
average of the section of the screen before that concerned for every section. It is good, though this 
difference is totaled for every frame, it asks for the location equivalent to CS2 of the location 1 where 
that value is large, i.e., CS of drawing 3 , and it is the change location of the same scene as the above of 
this. 

[0047] Return and each representation image extracted by doing in this way are supplied to drawing 1 at 
the epitome are recording section 21 and the alphabetic character data extraction section 23. The 
epitome are recording section 21 holds the set of the supplied representation image as an epitome 
corresponding to the image software concerned. In addition, the time of the recording start of the taken- 
out date and a original image is recorded on a file so that the relation between each acciunulated 
representation image and its original image may be known. Moreover, the information on the ejection 
location of the representation image is given to each representation image. This can be expressed with 
the number of progress seconds from there, a frame number, etc. on the basis of the recording start time 
of for example, a original image. 

[0048] From the supplied representation image, the alphabetic character data extraction section 23 
detects original image software to direct alphabetic data. There are some gestalten in the alphabetic data 
contained in image software. Therefore, detection here is also performed according to this gestalt. Since 
alphabetic data may be contained in one image software with two or more gestalten, it is good to also 
perform detection by two or more technique. 

[0049] (1) When the alphabetic character is incorporated into the image as an image, i.e., an image or 
dot data. 

In this case, an alphabetic character is detected with the application of the technique of Kanji OCR 
(optical alphabetic character read). Generally an alphabetic character is brought near and displayed on 
the lower part of an image, or right and left. Therefore, as long as precision may become coarse 
somewhat, the existence of an alphabetic character may be inspected about these parts. Then, 
simplification of processing and improvement in the speed can be performed. 
[0050] (2) When a character code is added. 

Like a teletext title, alphabetic data (title data) may be supplied by the character code as information on 
the skimmer of an image into the frame signal at die time of perpendicular feedback. In this case, it is 
easy. What is necessary is just to accumulate these character codes in the alphabetic character data 
accumulation section 28 in order with the positional information of an image. Furthermore, if it is made 
to develop, about the image software with which this character code is added, a representation image 
may not be extracted and you may extract only paying attention to the positional information of a 
character code and an image. 

[0051] Return and the user control unit 22 become drawing 1 from a keyboard, a mouse, etc. again. 
Following these actuation by the viewer, the retrieval control section 24 perfomis the ejection of the 
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ejection of the alphabetic data accumulated in the alphabetic character data accumulation section 28, the 
epitome corresponding to selected alphabetic data, and original image software, a display on a display 
26, etc. Moreover, 27 is a keyword attaching part and the keyword beforehand inputted by the viewer is 
held here. The retrieval control section 24 also performs registration processing of this keyword, and 
collating processing with these and the are recording data of the alphabetic character data accumulation 
section 28. 

[0052] Like the above-mentioned, indexes, such as a broadcast date, a time second, and a frame number, 
are given to each, and a original image and the alphabetic data summarized and detected are 
accumulated in it at the original image are recording section 12, the epitome are recording section 21, or 
the alphabetic character data accumulation section 28. These are the databases of image software itself 
Therefore, the method of the use is various. Although explanation of the above-mentioned rough 
actuation is overlapped, it will be as follows if an example is raised. 
[0053] (A) Whenever a viewer is the need, input and search a keyword. 

This search method is the most general database retrieval technique. If the theme which is an individual 
thing and a viewer wants for something to investigate specifically comes out, conceming (1) work, the 
theme will be driven in from the user control unit 22 supposing some language which will be expressed 
directly. 

[0054] (2) The retrieval control section 24 collates this keyword with the alphabetic data accumulated in 
the alphabetic character data accumulation section 28 following this, and inspects the existence of 
applicable data. Collating is carried out with the gestalt for which full coincidence and partial 
coincidence, others, and a viewer ask. 

(3) The retrieval control section 24 will indicate the table-of-contents information by list at a display 26, 
if the corresponding epitome and image software exist. If it held in media, such as DVD, the volume 
name showing the DVD etc., a title name, and a track number and others are indicated by list at a 
display 26. 

[0055] (4) A viewer chooses a desired thing out of this list If broadcasting hours, a DVD title, etc. make 
a key table-of-contents information currently taken out to the chart, the approach to the epitome for 
which it asks will become easy. The retrieval control section 24 takes out the epitome corresponding to 
the alphabetic data which the viewer chose from the epitome are recording section 21, and displays it on 
a display 26. 

[0056] Various methods of presentation of an epitome are also assumed. For example, the same with 
turning over a page, whenever the viewer pressed the key etc., every one representation image may be 
displayed. A representation image may be displayed on an automatic turning-over type one after another 
witti a certain amount of speed. Needless to say, there is story nature in image software. Therefore, even 
if it displays one after another at the early rate, a viewer can imderstand the contents of the epitome 
exactly unexpectedly. Functions, such as slow playback and a review, are also prepared and it is also 
good to enable it to use this if needed. 

[0057] (5) See a proper number of epitomes, or see to the last, and the image software or its part directs 
the thing relevant to the theme for which it asked, or playback of the original image software from the 
location which took out the representation image when it could judge that it was likely to be related. The 
epitome has magnitude of 1/100 from 1/10 of original image software, and much time amount is not 
taken to see. So, the direction made into two steps in this way can shorten the time of concentration to 
the original image for which it asks. 

[0058] Following directions of the above-mentioned viewer, the retrieval control section 24 reads the 
originzil image software after the epitome ejection location from the original image are recording section 
12 one by one, and displays it on a display 26. This processing is performed like regeneration with a 
video tape deck, others, and the usual visual equipment. Here, it is user-friendly when the key of a rapid 
traverse, a review, and a pause and others is prepared. 

[0059] (B) It is a Ust display about what registers the keyword beforehand and corresponded. 

In this case, the keyword showing the matter which a viewer wants to watch is previously accimiulated 

in the keyword attaching part 27. For example, if one day and a viewer do confirmation operation, a list 
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indication of the table-of-contents information on the epitome containing the alphabetic data which is in 
agreement with the keyword registered, or original image software will be given at a display 26 
following this. 

[0060] That is, in our working, it needs individually or the informational class to watch is restricted to 
some extent. Moreover, it is also rather troublesome to repeat and input the keyword same every day. 
Then, the keyword used for this seed fixed target is beforehand registered into the keyword attaching 
part 27 in this way. And what corresponds when a viewer wants is indicated by list at a display 26. For 
example, if you want to always grasp the image software about a patent and it will want, keywords, such 
as "a patent", a "industrial ri^t", and the "Patent Office", will be registered into the keyword attaching 
part 27, and corresponding image software will be ****(ed). If the identifier of a politician, a famous 
man, and a fibn star is registered, it can be begun immediately to find the image which he wants to 
know. 

[0061] (C) It is a list display about the alphabetic data accimiulated. 

For example, the alphabetic data accumulated is displayed on a display 26 in the order of the Japanese 
syllabary. A viewer chooses a desired thing out of it, and may be made to display the epitome and 
original image software corresponding to it. 

[0062] If a representation image to save, an epitome, image software, etc. come out after checking an 
epitome and image software by these various technique, the key non-illustrated "preservation" etc. will 
be operated. The retrieval control section 24 stores them in the request software are recording section 29 
following this. This becomes behind, and when required, it can refer to at any tiine. 
[0063] In addition, the audio playback approach is arbitrary. For example, what is necessary is just to 
reproduce at the same rate, when reproducing original image software. However, when reproducing an 
epitome, playback is impossible in the original **. Then, the voice data for several seconds near [ each ] 
a representation image extract location is taken out, for example, and it brings [ a little ] forward whether 
it is the same as the time of a original image in this, and reproduces. Moreover, after digitizing, it culls 
out suitably, and it may accumulate and it may be reproduced. 

[0064] A modification is described. Epitome playback may display four representation images and six 
on one screen of a display 26 collectively. The representation image has story nature as above- 
mentioned. Therefore, even if two or more these are displayed on a display 26 at once, a viewer can 
realize the contents easily. If it takes that there is a story into consideration, it can say that to do in this 
way is more intelligible. 

[0065] If it followed, for example, X representation images were displayed on the screen at once, the 
epitome concerned is simply reproducible by the time amount of 1 for X at the time of every one sheet. 
Since a duration is short and ends, display time may be lengthened about the display screen which 
compounded this representation image of X sheets to one sheet Thereby, an understanding of the image 
software concemed becomes easy fiirther. 

[0066] You may realize as equipment of dedication and this invention may be realized as an application 
program of a personal computer. An exclusive chip is manufactured, and it may include in a personal 
computer, or may include in a DVD player, a video tape deck, a game player, etc., or you may carry out 

[0067] About one image software, each representation image which makes change of a different element 
a motive may be extracted, and you may accumulate as a separate epitome. In the case of the contents 
check of record, each epitome can be reproduced and a subsequent check can be performed in the 
direction which is easy to understand. 

[0068] The magnetic tape of a video tape and others for home use may be used for the original image are 
recording section 12 and the epitome are recording section 21. When chart lasting time is insufficient, 
two or more decks (drive) are arranged. Are recording of original image software takes appropriate 
storage capacity. Therefore, it is thought that the above-mentioned cyclic method of the record is good. 
However, about an epitome and a character code, since necessary storage capacity is small, you may 
accumulate considerably for a long period of time. 

[0069] Although a check becomes a little difficult since original image software disappears by 
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overwrite, it can be grasped to some extent what the contents of the image software with which leaving 
alphabetic data and an epitome also corresponded to the keyword were. They can also be asked to a key 
to a broadcasting agency and data supply origin. Therefore, even if it makes it this operation format, a 
use is large as a database. 

[0070] It is better not to accumulate the thing without the need, since it grows also into the cause which 
increases a retrieval noise also about an epitome and a character code. For this reason, it is good also for 
processing in which will accumulate only what was decided for a viewer to inspect a part for what day, 
or the accumulated thing, and to save it, and the remainder will be eliminated on that spot (retrieval 
noise = data which have separated from the retrieval purpose although it corresponds to the keyword 
concerned). 

[0071] ^ 
[Effect of the Invention] As explained above, the alphabetic character which exists in image software 
took out and accumulated these paying attention to the property to express the contents of the image 
software directly, and made this invention available as a database. Therefore, when there is need always, 
a viewer is carrying out the keyword of a desired retrieval format, for example, the language currently 
used every day, etc., can pull out a request easily out of the image software supplied in large quantities, 
and can utilize these for work at an individual £q>plication. 



[Translation done.] 
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* NOTICES ♦ 

JPO and INPIT are not responsible for any 
damages caused k>y the use of this translation. 

1 .This document has been translated by computer. So the translation may not reflect the original 
precisely. 

2.**** shows the word which can not be translated. 
3. In the drawings, any words are not translated. 
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